[For maintainers] Suggested jobs to run (before merge)

run-slow: gemma3

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I'm guessing the context for this is that rope_scaling clobbering rope_parameters (the old behaviour) gave the right config for older models but breaks on upcoming models that might have specific configs for e.g. sliding_attention?

Yes, we've observed that rope_scaling was being applied to both full_attention and sliding_attention for the Gemma 3 checkpoints on HF Hub and in our attempts to re-convert them from the original Orbax checkpoint. The correct behavior is for rope_scaling to only be applied to full_attention if it exists in the config, and the sliding_attention config should always be default RoPE @ 10k.

Got it, thanks for the fix!